Start working on ability to have async_start_kernel in KernelManagers. #4126
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I would like to have ability to have async function in there in
particular because I need to start a kernel via slurm, and I don not kow
ahead of time which node it is going to be schedule on.
So far most of the solutions there are hacky and require having ssh
tunnels, which I would like to avoid. As so far the start_kernel
function already setup and connect ports it requires to be made async,
or deeply change the API.
I went with creating and checking for a second 'async_start_kernel' on
super, as it is MultiKernelManager in jupyter_clienc, and changing
`start_kernel` to be a coroutine function would be a break of API.
I have a patch on jupyter_client that itself check whether prorper kernel
manager start_kernel function are coroutine functions or not.
In the long run this should allow to progressively migrate
jupyter_client to use and async start_kernel, but we can't yet as anyway
we still support 2.7 on jupyter-Client.
Note that gen.maybe_future in tornado is deprecated, and it is
recommended to check for results and yield unknown objects:
Deprecated since version 4.3: This function only handles Futures,
not other yieldable objects. Instead of maybe_future, check for the
non-future result types you expect (often just None), and yield
anything unknown
The kernel_id is a Unicode Traitlet so unless subclass overwite that
should be fine.
Carreau
commented
Oct 22, 2018
| ) | ||
|
|
||
| sup = super(MappingKernelManager, self) | ||
| async_sk = getattr(sup, 'async_start_kernel', None) |
There was a problem hiding this comment.
Suggested change
| async_sk = getattr(sup, 'async_start_kernel', None) | |
| async_sk = getattr(sup, 'start_kernel_async', None) |
Carreau
commented
Oct 22, 2018
| sup = super(MappingKernelManager, self) | ||
| async_sk = getattr(sup, 'async_start_kernel', None) | ||
| if async_sk is not None: | ||
| res = super().async_start_kernel(**kwargs) |
There was a problem hiding this comment.
Suggested change
| res = super().async_start_kernel(**kwargs) | |
| res = super().start_kernel_async(**kwargs) |
Carreau
force-pushed
the
async-kernelmanagers
branch
from
a1e82fb to
c16b4aa
Compare
kevin-bates
reviewed
Oct 31, 2018
| if not isinstance(res, list): | ||
| kernels = res | ||
| else: | ||
| kernels = yield res |
There was a problem hiding this comment.
Should these statements be reversed? Seems like a list is the end goal here and anything but that will likely be a future to yield. I apologize if my understanding is not correct and, if not, perhaps a comment would help.
|
I believe you are right.
I'll fixed that next time I'm on my computer.
…On Wed, Oct 31, 2018, 13:38 Kevin Bates ***@***.*** wrote:
***@***.**** commented on this pull request.
------------------------------
In notebook/services/kernels/handlers.py
<#4126 (comment)>:
> @@ -29,7 +29,11 @@ class MainKernelHandler(APIHandler):
@gen.coroutine
def get(self):
km = self.kernel_manager
- kernels = yield gen.maybe_future(km.list_kernels())
+ res = km.list_kernels()
+ if not isinstance(res, list):
+ kernels = res
+ else:
+ kernels = yield res
Should these statements be reversed? Seems like a list is the end goal
here and anything but that will likely be a future to yield. I apologize if
my understanding is not correct and, if not, perhaps a comment would help.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#4126 (review)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAUezxR476_-Te-lhmhcQ3JnARhFJ8L3ks5uqgovgaJpZM4XxpT->
.
|
|
Closing as stale, thanks for experimenting with this @Carreau! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I would like to have ability to have async function in there in
particular because I need to start a kernel via slurm, and I don not kow
ahead of time which node it is going to be schedule on.
So far most of the solutions there are hacky and require having ssh
tunnels, which I would like to avoid. As so far the start_kernel
function already setup and connect ports it requires to be made async,
or deeply change the API.
I went with creating and checking for a second 'async_start_kernel' on
super, as it is MultiKernelManager in jupyter_clienc, and changing
start_kernelto be a coroutine function would be a break of API.I have a patch on jupyter_client that itself check whether prorper kernel
manager start_kernel function are coroutine functions or not.
In the long run this should allow to progressively migrate
jupyter_client to use and async start_kernel, but we can't yet as anyway
we still support 2.7 on jupyter-Client.
Note that gen.maybe_future in tornado is deprecated, and it is
recommended to check for results and yield unknown objects:
The kernel_id is a Unicode Traitlet so unless subclass overwite that
should be fine.